Papers with low-resource neural machine translation
Improving Back-Translation with Uncertainty-based Confidence Estimation (D19-1)
Copied to clipboard
| Challenge: | Despite the success of low-resource neural machine translation, there is a data scarcity problem in many languages . large-scale, high-quality, and widecoverage bilingual corpora do not exist for most language pairs . |
| Approach: | They propose to quantify confidence of NMT models based on model uncertainty . they propose to use uncertainty-based confidence measures to improve back-translation . |
| Outcome: | The proposed model outperforms conventional statistical machine translation (SMT) on Chinese-English and English-German translation tasks. |
kNN-TL: k-Nearest-Neighbor Transfer Learning for Low-Resource Neural Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Transfer learning is an effective technique for enhancing low-resource neural machine translation (NMT) however, these methods do not make use of the parent knowledge during the child inference, which may limit the translation performance. |
| Approach: | They propose a k-Nearest-Neighbor Transfer Learning approach which leverages the parent knowledge throughout the entire developing process of the child model. |
| Outcome: | The proposed approach outperforms strong baselines on four low-resource translation tasks. |
Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to transfer a pretrained NMT model to a new, unrelated language without shared vocabularies are limited to cognate languages. |
| Approach: | They propose to transfer a pretrained NMT model to a new, unrelated language without shared vocabularies by using cross-lingual word embedding and injecting artificial noises. |
| Outcome: | The proposed methods outperform multilingual joint training by a large margin in five low-resource translation tasks. |
A Systematic Study Reveals Unexpected Interactions in Pre-Trained Neural Machine Translation (2022.lrec-1)
Copied to clipboard
| Challenge: | Transfer learning is a promising direction for low-resource neural machine translation (NMT) but it introduces many new variables which are often selected through ablation studies, costly trial-and-error, or niche expertise. |
| Approach: | They conducted a three-factor experiment to examine how language similarity, pre-training dataset size and main dataset size interacted in their effect on performance in pre-trained transformer-based low-resource NMT. |
| Outcome: | The results suggest that systematic studies of interactions may be a promising long-term direction for guiding research in low-resource neural machine translation. |
Alternated Training with Synthetic and Authentic Data for Neural Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to synthesizing data in NMT focus on leveraging monolingual data in training. |
| Approach: | They propose alternated training with synthetic and authentic data to improve NMT models' performance. |
| Outcome: | The proposed approach improves Chinese-English and German-English translation tasks over strong baselines. |
A Novel Two-step Fine-tuning Framework for Transfer Learning in Low-Resource Neural Machine Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing transfer learning methods for neural machine translation use a well-trained translation model to initialize a child model with corresponding datasets. |
| Approach: | They propose a two-step fine-tuning framework for transfer learning in low-resource neural machine translation that adjusts the parent model to fit the child language by using the child source data. |
| Outcome: | The proposed framework improves on five low-resource translations on high-resolution languages. |
Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation (2020.acl-main)
Copied to clipboard
Aditya Siddhant, Ankur Bapna, Yuan Cao, Orhan Firat, Mia Chen, Sneha Kudugunta, Naveen Arivazhagan, Yonghui Wu
| Challenge: | Existing multilingual NMT approaches do not utilize the abundance of monolingual data, especially in low-resource languages. |
| Approach: | They propose to combine monolingual data with self-supervision to pre-train translation models and fine-tune on small amounts of supervised data. |
| Outcome: | The proposed approach improves translation quality of low-resource languages and zero-shot translation quality. |
COMET-QE and Active Learning for Low-Resource Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Using COMET-QE, we select sentences for low-resource neural machine translation. |
| Approach: | They propose a reference-free evaluation metric to select sentences for low-resource neural machine translation using Swahili, Kinyarwanda and Spanish. |
| Outcome: | The proposed method outperforms two variants of Round Trip Translation Likelihood and random sentence selection by up to 5 BLEU points on a 30k baseline. |
Meta-Learning for Low-Resource Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | In this paper, we propose to extend the recently introduced model-agnostic meta-learning algorithm for low-resource neural machine translation (NMT). |
| Approach: | They propose to extend the recently introduced meta-learning algorithm for low-resource neural machine translation (NMT) they frame low-Resource translation as a meta- learning problem where we learn to adapt to low-REsource languages based on multilingual high-resourced language tasks. |
| Outcome: | The proposed meta-learning algorithm outperforms the multilingual, transfer learning based approach and can train a competitive NMT system with only a fraction of training examples. |
ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing transfer learning methods for low-resource NMT are static, which simply transfer knowledge from a parent model to a child model once via parameter initialization. |
| Approach: | They propose a transfer learning method that can continuously transfer knowledge from the parent model during the training of the child model. |
| Outcome: | The proposed method can transfer knowledge from the parent model to the child model during the training of the child. |